Morphological Ending – based Strategies of Unknown Word Estimation for Statistical POS Urdu Tagger

Natural language processing has widely used Statistical based language models to solve disambiguation problems. Over the past decades different techniques regarding POS tagging have been proposed for English, European and East Asian languages. In this paper our focus is POS tagging for Urdu due to the infancy stage of Urdu language based tagging system. We have combined two approaches (Statistical and morphological ending based technique) to assign the appropriate syntactic categories, using Urdu language corpus as our experimental data. The process of our tagging system consists of two different stages. In first stage, we apply statistical based word level language model to compute the probability and assign the appropriate tag to the word. In second stage, we extract all the ambiguous and unknown words in the corpus and apply morphological ending based rules to resolve these discontinuities, which arise if statistical model fails to assign the appropriate tag to a given word. The development of this tagger is an initial step toward the Urdu POS tagging. The experimental results of the tagger show that the performance of the unknown word is improved when we add morphological ending based features with statistical model. Evaluation method employed shows the significance of experimental results and the effectiveness of morphological ending on statistical method. This is a pioneering work towards building a POS tagger for Urdu language through Morphological ending based strategies